Method and device for image processing by image division

ABSTRACT

Disclosed are a method and a device for processing an image by using image division capable of simultaneously minimizing deterioration of an image quality due to an error in a transmission line and further improving compression efficiency in performing compression and restoration of an image signal by using multiple processors. The method of processing an image by using image division according to an exemplary embodiment of the present disclosure includes dividing an image frame into multiple subblocks having a predetermined pixel size; constituting multiple macroblocks by sampling respective pixels positioned at an identical position within a predetermined number of subblocks; and constituting multiple slices by dividing the multiple macroblocks based on the positions of the sampled pixels within the subblock.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority from Korean PatentApplication No. 10-2011-0106781, filed on Oct. 19, 2011, and KoreanPatent Application No. 10-2012-0097005, filed on Sep. 3, 2012, with theKorean Intellectual Property Office, the disclosure of which isincorporated herein in its entirety by reference.

TECHNICAL FIELD

The present disclosure relates to a method and a device for processingan image by using image division, and more particularly to, a technologyfor dividing an image frame in order to process a high resolution imagein parallel by using a multi processor.

BACKGROUND

Screen resolution of a mobile device, such as a mobile phone, a PMP anda PDA, has been gradually improved, and mobile devices such as i-Phoneand tablet PC can also recently provide high quality video services. Ahome TV with screen resolution of 1920×1080 or more, that is, a level offull-HD, has been generalized together with the release of a digital TV,and thus a frame rate has been increased. An ultra HDTV (UHDTV) havingresolution of 4K or 8K (for example, 7680×4320) based on a horizontaldirection of the screen has attracted attention as a next-generation TV.

Image compression technologies capable of improving efficiency of imagecompression together with improvement of resolution of a display devicehave been developed and used. An amount of calculation for compressionand restoration of an image increases as an image size increases and aframe rate increases.

As a method for reducing power consumption while processing a largeamount of calculation, a multicore processor in which multiple CPUsexist in one chip was developed and has been universally used for a PC,a notebook computer and the like, and has been rapidly applied to asmart phone and a tablet PC.

In a general image compression scheme in the related art, in order toeffectively compress image data, an original image is divided intoseveral blocks having a predetermined size to be processed, and oneblock includes adjacent pixels on the image. For example, in a case ofthe H.264/AVC standard, an image is divided and processed by amacroblock unit, and the macroblock includes one luminance sample blockhaving a size of 16×16 and two chrominance sample blocks (a size of thechrominance sample block may vary according to a format of an inputimage). FIG. 1 is a diagram illustrating a method of dividing an imageinto macroblocks having a size of 16×16 according to the related art. Assuch, an entire image is divided into the macroblocks having a regularsize, so that an encoding process or a decoding process is performed bythe macroblock unit.

In order to prevent a decoded image from being deteriorated due toerrors of a transmission line, the image compression standard in therelated art uses a method of encoding an original image by dividing theoriginal image into multiple slices, and thus an error generated in oneslice is not propagated to another slice. For example, as illustrated inFIG. 2A, the MPEG-4 standard uses a method of constituting a slice bygrouping multiple macroblocks in the order of a raster scan. In theH.264/AVC standard, a slice is constituted in various methods, such as amethod of constituting a slice in an arbitrary order regardless of ascanning order as illustrated in FIG. 2B, a method of alternatelyallocating continuous macroblocks to different slice groups asillustrated in FIG. 2C, and a method of allocating macroblocks includedin a predetermined quadrangular area to different slice groups asillustrated in FIG. 2D.

In the meantime, the aforementioned method of dividing the image intothe slices or the slice groups may be used for the purpose of dividingand performing an image encoding or decoding process through multipleprocessors, as well as for reducing an influence of errors intransmission and decoding of the image. FIG. 3 is a diagram illustratinga method of dividing one image into multiple slices, simultaneouslyencoding the divided slices by multiple image encoding processors, andthen combining the encoded images into one bit stream.

However, in a case of using the method in the related art, it isimpossible to use compression information on an adjacent slice, so thatcompression efficiency is deteriorated compared to a case of encoding anentire image into a single slice, and an image quality is unavoidablydeteriorated in continuous predetermined areas of a decoded image whenan error is generated in a transmission line. When the continuousmacroblocks are allocated to the different slice groups in the method ofFIG. 2C or in a similar manner thereto, the deterioration of the imagequality due to the error may be dispersed to the entire image, butcompression efficiency is further deteriorated.

SUMMARY

The present disclosure has been made in an effort to provide a methodand a device for processing an image by using image division capable ofsimultaneously minimizing deterioration of an image quality due to anerror in a transmission line and further improving compressionefficiency in performing compression and restoration of an image signalby using multiple processors.

An exemplary embodiment of the present disclosure provides a method ofprocessing an image by using image division including dividing an imageframe into multiple subblocks having a predetermined pixel size,constituting multiple macroblocks by sampling respective pixelspositioned at an identical position within a predetermined number ofsubblocks; and constituting multiple slices by dividing the multiplemacroblocks based on the positions of the sampled pixels within thesubblock.

The method may further include encoding the multiple slices in parallelby using multiple image encoding processors and generating a bit streamby combining encoded data.

Another exemplary embodiment of the present disclosure provides a methodof processing an image by using image division, including: dividing animage frame into multiple subblocks having a predetermined pixel size;calculating an average value of pixels of each of the multiplesubblocks; and constituting one average value slice with pixels havingthe average value.

The method may further include: constituting multiple macroblocks bysampling respective pixels positioned at an identical position within apredetermined number of subblocks; and constituting multiple slices bydividing the multiple macroblocks according to the positions of thesampled pixels within the subblock, the multiple slices beingconstituted of difference value by calculating a difference valuebetween pixels included in each of the multiple slices and pixelsincluded in the average value slice.

The method may further include: encoding the average value slice and themultiple slices constituted of the difference values in parallel byusing multiple image encoding processors; and generating a bit stream bycombining the encoded data.

In the encoding, information on positions of the pixels constitutingeach of the slices within the subblock may be recorded in a header ofthe slice.

Yet another exemplary embodiment of the present disclosure provides adevice for processing an image by using image division, including: animage divider configured to divide an image frame into multiplesubblocks having a predetermined pixel size, constitute multiplemacroblocks by sampling respective pixels positioned at an identicalposition within a predetermined number of subblocks, and constitutemultiple slices by dividing the multiple macroblocks according topositions of the sampled pixels within the subblock; a multi processorcomprising multiple image encoding processors configured to encode themultiple slices in parallel; and a data combiner configured to generatea bit stream by combining the encoded data.

The image divider may calculate an average value of the pixels of eachof the multiple subblocks to further constitute one average value slicewith pixels having the average value, and may calculate a differencevalue between the pixels included in each of the multiple slices andpixels included in the average value slice to constitute the multipleslices with the difference values.

The multiple image encoding processors may encode the average valueslice and the multiple slices constituted of the difference values inparallel.

According to exemplary embodiments of the present disclosure, anencoding and a decoding are processed by dividing an image so thatadjacent pixels are included in different slices, respectively.Accordingly, even though an error is generated in a part of the slicesin a process of data transmission, the deterioration of the imagequality due to the error generation is dispersed to an entire image, sothat it is possible to remarkably reduce the evident deterioration ofthe image quality through peripheral pixels transmitted without anerror.

Compression efficiency is improved compared to a case in which the samenumber of slices is divided by the technology in the related art.

Even when only one among multiple slices is decoded, it is possible toobtain a reduced entire image, so that spatial scalability is furthereffectively provided.

The foregoing summary is illustrative only and is not intended to be inany way limiting. In addition to the illustrative aspects, embodiments,and features described above, further aspects, embodiments, and featureswill become apparent by reference to the drawings and the followingdetailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a method of dividing an image in therelated art.

FIGS. 2A to 2D are diagrams illustrating various methods of dividing anoriginal image into multiple slices.

FIG. 3 is a diagram illustrating a method of encoding an image by usingmultiple processors.

FIG. 4 is a flowchart illustrating a method of processing an image byusing image division according to an exemplary embodiment of the presentdisclosure.

FIG. 5 is a diagram for describing a method of dividing an imageaccording to the exemplary embodiment of FIG. 4 in detail.

FIG. 6 is a flowchart illustrating a method of processing an image byusing image division according to another exemplary embodiment of thepresent disclosure.

FIG. 7 is a configuration diagram illustrating a device for processingan image by using image division according to an exemplary embodiment ofthe present disclosure.

DETAILED DESCRIPTION

In the following detailed description, reference is made to theaccompanying drawing, which form a part hereof. The illustrativeembodiments described in the detailed description, drawing, and claimsare not meant to be limiting. Other embodiments may be utilized, andother changes may be made, without departing from the spirit or scope ofthe subject matter presented here.

FIG. 4 is a flowchart illustrating a method of processing an image byusing image division according to an exemplary embodiment of the presentdisclosure, and FIG. 5 is a diagram for describing a method of dividingan image according to the exemplary embodiment of FIG. 4 in detail.

Referring to FIGS. 4 and 5, the method of processing an image by usingimage division according to an exemplary embodiment of the presentdisclosure includes dividing an image frame into multiple subblockshaving a predetermined pixel size (S401), constituting multiplemacroblocks by sampling respective pixels positioned at an identicalposition within a predetermined number of subblocks (S403); andconstituting multiple slices by dividing the multiple macroblocks basedon the positions of the sampled pixels within the subblock (S405). Themethod of processing the image by using image division according toanother exemplary embodiment of the present disclosure may furtherinclude encoding the average value slice and the multiple slicesconstituted of the difference values in parallel by using multiple imageencoding processors (S407) and generating a bit stream by combiningencoded data (S409).

In the present exemplary embodiment, it is assumed that one macro blockhas a 16×16 pixel size. However, it is apparent that a size of themacroblock may vary according to a format, resolution and the like ofthe image.

In step S401, the original image is divided so that M×N pixels of theoriginal image constitute one subblock. In the present exemplaryembodiment, M=N=2, and as illustrated in FIG. 5, positions of fouradjacent pixels within the one subblock are represented by a, b, c andd.

In step S403, the multiple macroblocks are constituted by sampling thepixels positioned at the same position within 16×16 subblocks. That is,when a sampling is performed in a horizontal direction by M:1 and in avertical direction by N:1, pixels are alternately selected one by one inthe horizontal and vertical directions under the condition of M=N=2, sothat the respective pixels positioned at the adjacent positions a, b, cand d within the respective subblocks are divided to constitute thedifferent macroblocks. Through the method, the pixels positioned at theposition a are collected to constitute macroblocks a0, a1, . . . , andsimilarly, the pixels positioned at the positions, b, c and d arecollected to constitute macroblocks b0, b1, . . . , c0, c1, . . . , andd0, d1, . . . .

In step S405, the slice consists of the multiple macroblocks includingthe pixels positioned at the same position. That is, slice A isconstituted by collecting the macroblocks a0, a1, . . . , and slices B,C and D are constituted in the same manner, respectively.

Next, in step S407, the slices A, B, C and D are simultaneously encodedin parallel by using the multiple image encoding processors, and in stepS409, the encoded data is combined to generate one bit stream. Therespective slices may be combined and transmitted without being limitedby the order.

In this case, in order to achieve the accurate decoding of the bitstream, information on the positions of the pixels included in therespective slices within the subblock may be recorded in a header of theslice in the process of the encoding. Otherwise, necessary informationmay be removed from the header of the slice by pre-defining values of Mand N and also pre-defining a transmission order of the slices. In asubsequent decoding process, decoding results of the respective slicesmay be re-combined by using the recorded information or the pre-definedorder so that the pixels included in the respective slices may bedisplayed at proper positions on the image.

In the meantime, according to the present disclosure, spatialscalability may be effectively supported in one bit stream. That is,according to the exemplary embodiment, since the respective slicesequally have the spatial information on the image, even if any one amongthe slices A, B, C and D is decoded, it is possible to obtain a reducedentire image.

FIG. 6 is a flowchart illustrating a method of processing an image byusing image division according to another exemplary embodiment of thepresent disclosure.

Referring to FIG. 6, a method of processing an image according toanother exemplary embodiment of the present disclosure includes dividingan image frame into multiple subblocks having a predetermined pixel size(S601), calculating an average value of pixels of each of the multiplesubblocks (S603), constituting one average value slice with the pixelshaving the calculated average value (S605), constituting multiplemacroblocks by sampling respective pixels positioned at the sameposition within the predetermined number of subblocks (S607), andconstituting multiple slices by dividing the multiple macroblocksaccording to the positions of the sampled pixels within the subblock,the multiple slices being constituted of difference values bycalculating a difference value between the pixels included in each ofthe multiple slices and the pixels included in the average value slice(S609). The method of processing the image by using image division mayfurther include encoding the average value slice and the multiple slicesconstituted of difference values in parallel by using multiple imageencoding processors (S611) and generating a bit stream by combiningencoded data (S613).

In the present exemplary embodiment, as an applied example in whichspatial scalability is further evolved from the exemplary embodiment ofFIGS. 4 and 5, a separate average value slice constituted of pixelshaving an average value of the pixels included in each of the subblocksillustrated in FIG. 5 may be generated, and the remaining slices may beconstituted of the difference values from the pixels included in theaverage value slice in order to improve encoding efficiency. In thiscase, in a decoding process, a restored image with a small size may beobtained by decoding only the average value slice, and when theremaining slices are additionally decoded, the image having the samesize as that of the original image may be obtained.

FIG. 7 is a configuration diagram illustrating a device for processingan image by using image division according to an exemplary embodiment ofthe present disclosure.

Referring to FIG. 7, a device for processing an image by using imagedivision according to an exemplary embodiment of the present disclosureincludes an image divider 701, a multi processor 703 and a data combiner705.

The image divider 701 may divide an image frame into multiple subblockshaving a predetermined pixel size, constitute multiple macroblocks bysampling respective pixels positioned at an identical position within apredetermined number of subblocks, and constitute multiple slices bydividing the multiple macroblocks according to positions of the sampledpixels within the subblock.

The image divider 701 may calculate an average value of the pixels ofeach of the multiple subblocks and further constitute one average valueslice with pixels having the calculated average value. In this case, theimage divider 701 may also constitute the multiple slices withcalculated difference values by calculating a difference value betweenthe pixels included in each of the multiple slices and the pixelsincluded in the average value slice.

The multi processor 703 includes multiple image encoding processors P_A,P_B, . . . and P_X in order to simultaneously encode the multiple slicesin parallel. The multiple image encoding processors P_A, P_B, . . . andP_X may record information on positions of the pixels constituting eachof the slices within the subblock in a header of the slice in theencoding. The multi processor 703 includes multiples cores in one chip,so that the multi processor 703 may be implemented in a form like amulticore processor which performs different calculations in parallel.

The data combiner 705 generates one bit stream by combining data that isencoded in parallel. A combination order of the parallel data may bedetermined in a defined order according to a standard or the like of animage or may be arbitrarily determined.

More detailed functions of the respective elements and effects thereofare the same as those described with reference to FIGS. 4 to 6.

From the foregoing, it will be appreciated that various embodiments ofthe present disclosure have been described herein for purposes ofillustration, and that various modifications may be made withoutdeparting from the scope and spirit of the present disclosure.Accordingly, the various embodiments disclosed herein are not intendedto be limiting, with the true scope and spirit being indicated by thefollowing claims.

What is claimed is:
 1. A method of processing an image by using imagedivision, comprising: dividing an image frame into multiple subblockshaving a predetermined pixel size; constituting multiple macroblocks bysampling respective pixels corresponding to an identical position,differing from each of others of adjacent positions, within respectivesubblocks of the multiple subblocks; and constituting multiple slices byclassifying the multiple macroblocks based on a criterion of theidentical position, differing from the others of the adjacent positions,within respective subblocks of the multiple subblocks.
 2. The method ofclaim 1, further comprising: encoding the multiple slices in parallel byusing multiple image encoding processors; and generating a bit stream bycombining the encoded data.
 3. The method of claim 2, wherein in theencoding, information on positions of the pixels constituting each ofthe slices within the subblock is recorded in a header of the slice. 4.The method of claim 1, wherein the macroblock has a 16×16 pixel size.